Image processing device

ABSTRACT

An image processing device for generating both of a distance image and a gray image from an electrical output of a light receiving element on the precondition that a light intensity-modulated at a modulation frequency is irradiated into a target space. This device has an image generator for generating the distance image having pixel values, each of which provides a distance value between an object in the target space and the device, in accordance with a phase difference between the irradiated light and the received light, and the gray image having pixel values, each of which provides a gray value of the object, in accordance with an intensity of the received light. By use of an output of the image generator, an outline of the object can be extracted.

TECHNICAL FIELD

The present invention relates to an image processing device forextracting spatial information form a target space, into which anintensity-modulated light is irradiated.

BACKGROUND ART

In the past, various types of spatial information detecting devices formeasuring distance information of an object or extracting an outline ofthe object from an output of image pickup means have been proposed. Forexample, Japanese Patent Publication Laid-open No. 11-284997 discloses atechnique of extracting the outline of the object from a gray imagegenerated by use of an image sensor. In addition, Japanese PatentPublication Laid-open No. 64-10108 discloses a technique of determininga distance with the object by irradiating a spot-like or linear lightpattern to an object, receiving a light reflected from the object by aposition sensitive detector (PSD), and converting an output of theposition sensitive detector into the distance in accordance with atriangular surveying method. In addition, PCT Gazette WO03/085413discloses a spatial information detecting device for detecting spatialinformation such as distance from an electrical output corresponding toan intensity of received light, which is obtained by irradiating a lightintensity-modulated at an emission frequency to a target space, andreceiving the light reflected from an object in the target space.

By the way, a greater amount of spatial information can be obtained byusing both of the gray image and the distance information. However,according to the conventional techniques, since each of gray values ofthe gray image and a corresponding distance value are not obtained fromthe same pixel, a treatment of associating each of positions in the grayimage with corresponding distance value is separately needed. Forexample, since a light is scanned in a target space in the apparatususing the triangular surveying method, a relatively large time lagbetween the generation of the gray image and the generation of thedistance information occurs, so that the associating treatmenttherebetween becomes complex. In addition, when using both of the devicefor generating the gray image such as a TV camera with a CCD imagesensor and the device for detecting the distance information such as theposition sensitive detector, an increase in size and cost of the wholeapparatus also becomes a problem.

SUMMARY OF THE INVENTION

Therefore, a primary concern of the present invention is to provide animage processing device having the capability of generating both of adistance image and a gray image by irradiating a lightintensity-modulated at a modulation frequency to a target space, andreceiving the light reflected from an object in the target space.

That is, the image processing device of the present invention comprises:

a light source configured to irradiate a light intensity-modulated at amodulation frequency to a target space;

a light receiving element such as photoelectric converter configured toreceive the light reflected from an object in the target space andgenerate an electrical output corresponding to an intensity of thereceived light; and

an image generator configured to generate a distance image having pixelvalues, each of which provides a distance value between the object andthe image processing device, in accordance with a phase differencebetween the light emitted from the light source and the light receivedby the light receiving element, and a gray image having pixel values,each of which provides a gray value of the object, in accordance withthe intensity of the received light.

According to the present invention, the gray image and the distanceimage of the object can be obtained from the electrical outputcorresponding to the intensity of the light received by the lightreceiving element at a time. In addition, since each of the gray valuesof the gray image and a corresponding distance value of the distanceimage are obtained from the same pixel, no treatment of associating eachof positions in the gray image with the corresponding distance value isneeded. Consequently, it is possible to obtain a greater amount of thespatial information by using both of the gray image and the distanceimage without performing such a complex associating treatment.Furthermore, as compared with the case of combining a conventional imagepickup device for generating only the gray image with a conventionaldistance measuring device for extracting the distance information, thereare another advantages of downsizing the device as a whole, andachieving a cost reduction.

In the present invention, it is preferred that the image processingdevice further comprises a differentiator configured to generate adistance differential image having pixel values, each of which providesa distance differential value, from the distance image, and a graydifferential image having pixel values, each of which provides a graydifferential value, from the gray image, and an outline extractorconfigured to extract an outline of the object by use of the distancedifferential image and the gray differential image. In this case, it ispossible to reduce amounts of noises, and clearly extract the outline ofthe object, as compared with the case of using only the gray image.

It is further preferred that the image generator generates the grayimage in a time-series manner, and the image processing device furthercomprises a differentiator configured to generate a gray differentialimage having pixel values, each of which provides a gray differentialvalue, from the gray image, and an object detector configured to detectthe object by use of the gray differential value and the distance value.In this case, a region with a large difference in contrast can be easilyseparated from another region with a relatively small difference incontrast in the target space. Therefore, it is effective to extract theoutline of the object under a high contrast condition between the objectand the background in the target space. In addition, the outline of theobject within a desired distance range can be obtained from the distancevalues of the distance image corresponding to the region extracted bythe using the gray differential image.

It is also preferred that the object detector generates a differenceimage between a pair of gray differential images, which are generatedfrom two gray images obtained at different times, extracts a regionwhere each of pixel values is not smaller than a threshold value in thedifference image, and then detects the region as the object when arepresentative value of the pixel values of the distance imagecorresponding to the region is within a predetermined range. In thiscase, only a region where a brightness change occurs in the target spacecan be extracted. In addition, since the region where each of the pixelvalues is smaller than the threshold value is removed, a region of theobject traveled between the different times at which the two gray imagesare generated can be extracted. Furthermore, it is possible toaccurately separate the object region from the background depending onthe distance by use of the difference image derived from the gray imagesand the distance image.

In addition, it is preferred that the object detector generates aplurality of difference images, each of which is a difference betweentwo of at least three gray differential images generated from at leastthree gray images obtained at different times, extracts a region(s)where each of pixel values is not smaller than a threshold value withrespect to each of the difference images to obtain binary images,performs a logical operation between each of pixel values of one of thebinary images and a corresponding pixel value of another one of thebinary images to extract a common region therebetween, and detects thecommon region as the object when a representative value of the pixelvalues of the distance image corresponding to the common region iswithin a predetermined range. In this case, there is an advantage ofextracting a silhouette of the object traveling in the target space,while almost removing the background. Moreover, it is possible toaccurately separate the object region from the background depending onthe distance by use of the difference images derived from the grayimages and the distance image.

As a preferred embodiment of the present invention, the image processingdevice further comprises a measuring-point determining unit configuredto determine a plurality of measuring points on the object in the grayimage generated by the image generator; and a distance calculatorconfigured to calculate an actual distance between two of the measuringpoints on the object by use of the distance value of the pixelcorresponding to each of the measuring points in the distance imagegenerated by the image generator. In this case, it is possible to easilydetermine the actual size of a required portion of the object, ascompared with the case of combining the conventional image pickup devicefor generating only the gray image with the conventional distancemeasuring device for extracting the distance information, and thenperforming the treatment of associating each of positions in the grayimage with the corresponding distance value.

As a further preferred embodiment of the present invention, the imageprocessing device further comprising a shape estimating unit configuredto estimate a 3D model of the object from at least one of the distanceimage and the gray image generated by the image generator, and a volumeestimating unit configured to estimate a volume of the object inaccordance with outputs of the shape estimating and the distancecalculator described above. In particular, when a monitor for displayingthe gray image generated by the image generator is provided, and themeasuring-point determining unit comprises a position designatorconfigured to allow a user to appoint desired measuring points on theobject displayed on the monitor by touching a screen of the monitor, theactual distance between two of the desired measuring points appointed bythe position designator can be calculated by the distance calculator. Inthis image processing device, even though the light receiving elementreceives the light reflected by a three-dimensional object from only onedirection, the 3D information such as shape and volume of the object canbe relatively accurately estimated by using both of the distance imageand the gray image. In addition, the actual size of a desired portion ofthe object can be easily calculated.

As another preferred embodiment of the present invention, the imageprocessing device further comprises an object extractor configured toextract the object having a predetermined shape from the gray imagegenerated by the image generator, and the measuring-point determiningunit determines a plurality of measuring points on the object extractedby the object extractor, and the distance calculator calculates theactual distance between two of the determined measuring points. In thiscase, since it is not needed to allow the user to designate themeasuring points, the actual size of the predetermined portion of theobject can be automatically calculated. In addition, it is possible toreduce variations in measurement results of the actual size, as comparedwith the case that the measuring points are designated every time by theuser.

As another preferred embodiment of the present invention, the imageprocessing device further comprises a reference-pixel detectorconfigured to detect, as a reference pixel, the pixel having a minimumdistance value in a predetermined region in the distance image; a pixelextractor configured to set a specific region including the referencepixel in the distance image, and extract a group of pixels each havingthe distance value within a predetermined range from the specificregion; and an exposure controller configured to control a sensitivityof the light receiving element in accordance with the gray image havingthe pixels, each of which has a one-to-one correspondence with one ofthe pixels extracted by the pixel extractor. In this case, the lightreceiving element can be automatically controlled to a correct exposureirrespective of the brightness of the target space or the background ofthe object. Therefore, this image processing device is preferably usedfor a TV interphone.

These and additional objects and advantages of the present inventionwill become more apparent from the best mode for carrying out theinvention explained below, referring to the attached drawings.

BRIEF EXPLANATION OF THE DRAWINGS

FIG. 1 is a block diagram of an image processing device according to afirst embodiment of the present invention;

In FIGS. 2A to 2C, FIG. 2A is a diagram illustrating a Time-of-Flightmethod, and FIGS. 2B and 2C show timings of applying a control voltageto an electrode(s) of a light receiving element;

FIGS. 3A and 3B are schematic cross-sectional views illustrating asensitivity control method of the light receiving element;

FIG. 4 is a plan view of the light receiving element;

FIG. 5 is a block diagram illustrating another light receiving element;

FIGS. 6A and 6B are schematic diagrams illustrating charge generationand holding operations of the image processing device;

FIGS. 7A and 7B are schematic diagrams illustrating of another chargegeneration and holding operations of the image processing device;

FIG. 8 shows a 3×3 pixel arrangement used to determine a distancedifferential value;

FIG. 9 is a block diagram of an image processing device according to asecond embodiment of the present invention;

FIG. 10 is a diagram illustrating a method of extracting an outline ofan object traveling in a target space;

FIG. 11 is a block diagram of an image processing device according to athird embodiment of the present invention;

FIG. 12 is a block diagram of an image processing device according to afourth embodiment of the present invention;

FIG. 13 is a schematic diagram illustrating an operation of the imageprocessing device;

FIGS. 14A and 14B are schematic diagrams of distance images of an objectgenerated by the image processing device;

FIG. 15 is a block diagram of an image processing device according to afifth embodiment of the present invention; and

FIG. 16 is a schematic diagram illustrating an operation of the imageprocessing device.

BEST MODE FOR CARRYING OUT THE INVENTION First Embodiment

As shown in FIG. 1, an image processing device of the present embodimentcomprises a light source 1 for irradiating a light into a target space,light receiving element 2 for receiving the light reflected from anobject M such human in the target space, control unit 3 for the lightreceiving element, image generator 4 for generating a distance image anda gray image from an output of the light receiving element 2,differentiator 50 for generating a distance differential image from thedistance image and a gray differential image from the gray image, and anoutline extractor 52 for extracting an outline of the object M by use ofthe distance differential image and the gray differential image.

The present invention is based on the premise that a distance betweenthe light source 1 and the object M is determined from a time of flight,which is defined a time period elapsed between the irradiation of thelight from the light source and the reception of the light reflectedfrom the object by the light receiving element 2. Since the time offlight is extremely short, a light intensity-modulated at a requiredmodulation frequency is irradiated from the light source 1. Therefore,the distance can be determined by using a phase difference between theintensity-modulated light emitted from the light source 1 and the lightreceived by the light receiving element 2.

The “Time-of-flight” method is described in U.S. Pat. No. 5,656,667 andPCT Gazette No. WO03/085413. Therefore, the principle is brieflyexplained in the present specification. For example, as shown in FIG. 2,when an intensity of the light emitted from the light source 1 changes,as shown by the curve “S1”, and the intensity of the light received bylight receiving element 2 changes, as shown by the curve “S2”, theintensity of the received light can be detected at each of fourdifferent phases (0°, 90°, 180°, 270°) to obtain four intensities (A0,A1, A2, A3). Since it is impossible to detect the intensity of lightreceived at just the moment of the each of the phases (0°, 90°, 180°,270°), each of the intensities (A0, A1, A2, A3) practically correspondsto the intensity of the light received within a short time width “Tw”.On the assumption that the phase difference “ψ” does not change withinthe modulation period, and there is no change in light extinction ratioin the time period between the irradiation of the light from the lightsource 1 and the reception of the light reflected from the object M, thephase difference “ψ” can be represented by the following equation (1):Ψ=tan⁻¹{(A2−A0)/(A1−A3)}  (1)In the present invention, the intensities of the received light may bedetected at different phases other than the four different phases (0°,90°, 180°, 270°) spaced from each other by 90 degrees.

As the light source 1, for example, an array of light emitting diodes(LED) or a combination of a semiconductor laser and a divergent lens canbe used. The light source 1 is driven by a modulation signal with arequired modulation frequency, which is provided from the control unit3, to emit the light intensity-modulated by the modulation signal. As anexample, the light intensity-modulated by a sine wave of 20 MHz isirradiated to the target space. Alternatively, the intensity-modulationmay be performed by use of another waveform such as a triangular wave ora saw-tooth wave.

The light receiving device 2 is composed of a plurality of photoelectricconverters 20, each of which receives the light reflected from theobject in the target space at a light receiving surface, and generatesamounts of electric charges corresponding to the intensity of thereceived light, sensitivity controller 22 for controlling thesensitivity of each of the photoelectric converters, charge collectingportion 24 for collecting at least part of the electric chargesgenerated by the photoelectric converter, and a charge ejecting portion26 for outputting the electric charges from the charge collectingportion. In the present embodiment, the amounts of the received light(A0, A1, A2, A3) are determined at the four timings synchronized with achange in intensity of the light emitted from the light source 1 toobtain the distance between the image processing device and the object.These timings are controlled by the control unit 3, as described later.Due to small amounts of electric charges generated by each of thephotoelectric converters 20 in one cycle of the intensity change of thelight emitted from the light source, it is preferred that the electriccharges are collected in plural cycles of the intensity change of theemitted light. For example, the photoelectric converters 20, sensitivitycontrollers 22 and the charge collecting portions 24 are provided as asingle semiconductor device. The charge ejecting portion 26 may have asubstantially same structure as a vertical transfer portion or ahorizontal transfer portion of a conventional CCD image sensor.

The intensity-modulated light is reflected by the object M, and then thereflected light is incident on the photoelectric converters 22 through arequired optical system 5. As shown in FIGS. 3A and 3B, each of thephotoelectric converters 20 is composed of a doped semiconductor layer11 and, an insulating film 12 such as an oxide film formed on a generalsurface of the doped semiconductor layer 11. A plurality of controlelectrodes 13 are formed on the doped semiconductor layer 11 through theinsulating film 12. For example, a matrix arrangement of 100×100photoelectric converters 22 can be used as an image sensor.

This type of the light receiving element 2 can be obtained by forming amatrix pattern of the photoelectric converters 20 in a singlesemiconductor substrate. In each column of the matrix pattern of thephotoelectric converters 20, the doped semiconductor layer 11 iscommonly used as a vertical transfer portion to transfer the electriccharges (electrons “e”) in the columnar direction. On the other hand,the electric charges provided from an end of the semiconductor layer 11of each column of the matrix pattern are transferred in the rowdirection through a horizontal transfer portion. For example, as shownin FIG. 4, the light receiving element 2 has an image pickup portion Daformed by the matrix pattern of the photoelectric converters 20 and anaccumulating portion Db with light shielding disposed adjacent to theimage pickup region Da. The electric charges collected in theaccumulating portion Db are transferred into the horizontal transferportion Th. The charge ejecting portion 26 includes a function oftransferring the electric charges in the vertical direction as well asthe horizontal transfer portion Th. The charge collecting portion 24described above means the function of collecting the electric charges inthe image pickup region Da, but not the accumulating portion Db. Inother words, the accumulating portion Db belongs to the charge ejectingportion 26. These vertical and horizontal transfer portions are similarto configurations of the conventional frame transfer (FT) CCD imagesensor. Therefore, further detailed explanation is omitted.

The optical system 5, through which the light reflected from the objectM is incident on the light receiving element 2, determines a visual axisconnecting between each of the photoelectric converters 20 and acorresponding point on the object. Generally, the optical system 5 isformed such that a light axis is orthogonal to a plane of the matrixarrangement of the photoelectric converters 20. For example, when acenter of the optical system 50 is defined as an original point, and anorthogonal coordinate system is set by vertical and horizontaldirections in the plane and the light axis, the optical system isdesigned such that an angle (i.e., azimuth angle and elevation angle)obtained by describing a position on the object M in the target spacewith a spherical coordinate corresponds to each of the photoelectricconverters 20. Therefore, when the light reflected from the object M isincident on one of the photoelectric converters 20 through the opticalsystem 5, a direction of the visual axis connecting between thephotoelectric converter and the corresponding position on the objectwith respect to the light axis as a reference direction can bedetermined by use of the position of the photoelectric converter 20.

The light receiving device 2 with the above-described structure is knownas a MIS (Metal-Insulator-Semiconductor) device. However, the lightreceiving device 2 of this embodiment is different from the conventionalMIS device in that a plurality of control electrodes 13 (for example,five control electrodes shown in FIG. 3A) are formed on each of thephotoelectric converters 20. The insulating film 12 and the controlelectrodes 13 are made of a translucent material. When a light isincident on the doped semiconductor layer 11 through the insulating film12, electric charges are generated in the doped semiconductor layer 11.The doped semiconductor layer 11 shown in FIG. 3A is an n-typesemiconductor layer. Therefore, the generated electric charges areelectrons (e).

According to the sensitivity controller 22 of this embodiment, theamounts of the electric charges generated by the photoelectric converter20 can be controlled by changing an area of a light receiving region ofthe photoelectric converter (i.e., light receiving area). For example,when a control voltage (+V) is applied to three of the five controlelectrodes 13, as shown in FIG. 3A, a potential well (depletion layer)14 is formed over a region corresponding to the three control electrodesin the doped semiconductor layer 11, as shown by the dotted line in FIG.3A. When the light is incident on the photoelectric converter 20 havingthe formed potential well 14, parts of electrons generated in the dopedsemiconductor layer 11 are captured in the potential well, and thebalance of the generated electrons are lost by direct recombination withholes at a deep portion of the doped semiconductor layer 11.

On the other hand, when the control voltage (+V) is applied to thecenter one of the five control electrodes 13, the potential well 14 isformed over a region corresponding to the one electrode in the dopedsemiconductor layer 11, as shown by the dotted line in FIG. 3B. Since adepth of the potential well 14 of FIG. 3A is equal to the depth of thepotential well 14 of FIG. 3B, the size of the potential well of FIG. 3Ais larger than that of the potential well of FIG. 3B. Therefore, whenthe same light amount is supplied into each of the light receivingdevices 2 of FIGS. 3A and 3B, the potential well of FIG. 3A can outputlarger amounts of electric charges as signal charges. This means thatthe light receiving element 2 has a higher sensitivity under thecondition of FIG. 3A, as compared with the case of FIG. 3B.

Thus, by changing the number of the control electrodes 13, to which thecontrol voltage is applied, a size of the potential well 14 in adirection along the general surface of the doped semiconductor layer 11(in other words, the size of the charge collecting portion 24 in thelight receiving surface) can be controlled to achieve a desiredsensitivity of the light receiving element 2.

Alternatively, the sensitivity of the light receiving element 2 may becontrolled by changing a ratio of amounts of the electric charges givento the charge collecting portion 24 relative to the amounts of electriccharges generated by the photoelectric converter 20, as disclosed in PCTGazette WO03/085413. In the case of using this control method, it ispreferred to perform one of techniques of controlling only a flow of theelectric charges from the photoelectric converter 20 to the chargecollecting portion 24, controlling only a flow of electric charges fromthe photoelectric converter to a charge discarding portion, andcontrolling both of these flows of the electric charges. As an example,the case of controlling the flows of electric charges from thephotoelectric converter to the charge collecting portion and the chargediscarding portion is explained below.

As shown in FIG. 5, the light receiving element 2 used in this controlmethod has a gate electrode 23 formed between each of the photoelectricconverters 20 and a corresponding charge collecting portion 24, and acharge discarding portion 27 commonly used by the photoelectricconverters 20. By changing a first control voltage applied to the gateelectrode 23, amounts of the electric charges traveling from one of thephotoelectric converters 20 to the corresponding charge collectingportion 24 can be controlled. In addition, amounts of electric chargestraveling from one of the photoelectric converters 20 to the chargediscarding portion 27 can be controlled by changing a second controlvoltage applied to a control electrode 25 for the charge discardingportion 27. In this case, for example, an interline transfer (IT) type,frame transfer (FT) type, or frame interline transfer (FIT) type CCDimage sensor having an overflow drain can be used as the light receivingelement 2 with the sensitivity controller 22.

Next, a method of determining the four intensities (A0, A1, A2, A3) ofthe received light by controlling the sensitivity of the photoelectricconverters 20 to obtain the distance information with the object M isexplained. As described above, the control unit 3 controls the controlvoltage applied to the control electrodes 13 to change the area of thepotential well 14 formed in the photoelectric converter 20, i.e., thesize of the charge collecting portion 24. In the following explanation,as shown in FIGS. 6A and 6B, six control electrodes 13 for a pair ofphotoelectric converters 20 providing one pixel are numbered as (1) to(6). Therefore, one of the pair of photoelectric converters 20 has thecontrol electrodes (1) to (3), and the other one has the controlelectrodes (4) to (6).

For example, electric charges corresponding to each of the intensities(A0, A2) of the received light can be alternately generated by use ofthe pair of photoelectric converters 20 providing one pixel. In the caseof generating the electric charges corresponding to the intensity (A0),the potential well 14 having the large area can be obtained by applyinga constant control voltage to all of the control electrodes (1) to (3)of one of the photoelectric converters 20, as shown in FIG. 6A At thistime, with respect to the other photoelectric converter 20, the controlvoltage is applied to only the center electrode (5) of the controlelectrodes (4) to (6) to obtain the potential well 14 having the smallarea. The large potential well 14 formed in the photoelectric converter20 having the control electrodes (1) to (3) is in a charge generationperiod with a high sensitivity state, and the small potential well 14formed in the other photoelectric converter 20 having the controlelectrodes (4) to (6) is in a charge holding period with a lowsensitivity state. Under this condition, the electric chargescorresponding to the intensity (A0) can be collected in the largepotential well 14 of the photoelectric converter having the controlelectrodes (1) to (3).

On the other hand, when generating the electric charges corresponding tothe intensity (A2), the potential well 14 having the large area can beobtained by applying the constant control voltage to all of theelectrodes (4) to (6) of one of the photoelectric converters 20, asshown in FIG. 6B. At this time, with respect to the other photoelectricconverter 20, the control voltage is applied to only the centerelectrode (2) of the control electrodes (1) to (3) to obtain thepotential well 14 having the small area. The large potential well 14formed in the photoelectric converter 20 having the control electrodes(4) to (6) is in a charge generation period with a high sensitivitystate, and the small potential well 14 formed in the other photoelectricconverter 20 having the control electrodes (1) to (3) is in a chargeholding period with a low sensitivity state. Under this condition, theelectric charges corresponding to the intensity (A2) can be collected inthe large potential well 14 of the photoelectric converter 20 having thecontrol electrodes (4) to (6). Thus, by alternately repeating theformation of the large potential well 14 in the photoelectric converter20 having the control electrodes (1) to (3) and the formation of thelarge potential well 14 in the photoelectric converter 20 having thecontrol electrodes (4) to (6), the electric charges corresponding toeach of the intensities (A0, A2) of the received light can be obtained.

The timing of applying the control voltage to the control electrodes togenerate the electric charges corresponding to each of the intensities(A0) and (A2) are shown in FIGS. 2B and 2C, in which a hatching regiondesignates that the control voltage is applied to the controlelectrodes. The electric charges corresponding to each of theintensities (A1, A3) of the received light can be alternately generatedby use of the pair of the photoelectric converters 20 providing onepixel according to a substantially same method described above exceptthat the timing of applying the control voltage to the controlelectrodes is shifted by 90 degrees with respect to the phase of themodulation signal. Thus, the control unit 3 controls the timing ofapplying the control voltage to the control electrodes and the number ofcontrol electrodes, to which the control voltage is applied. In otherwords, to determine the phase difference between the light irradiatedfrom the light source 1 into the target space and the light received bythe light receiving element 2, the sensitivity of the light receivingelement is controlled at the timing synchronized with the period of themodulation signal for driving the light source 1 by the control unit 3.That is, high and low sensitivity states of the light receiving element2 are alternately repeated by a repetition cycle synchronized with theperiod of the modulation signal by the control unit 3.

After the electric charges corresponding to the intensity (A0) arecollected in the large potential well 14 shown in FIG. 6A, and theelectric charges corresponding to the intensity (A2) are collected inthe large potential well 14 shown in FIG. 6B, these electric charges areoutput from the charge ejecting unit 26. Similarly, the electric chargescorresponding to each of the intensities (A1) and (A3) are collected,and then output from the charge ejecting unit 26. Thus, by repeating theabove procedures, it is possible to obtain the electric chargescorresponding to each of the four intensities (A0, A1, A2, A3) of thereceived light, and determine the phase difference “ψ” by use of theabove equation (1).

It is also preferred that the control voltage applied to the controlelectrodes 13 in the charge generation period is greater than thecontrol voltage applied to the control electrode(s) in the chargeholding period. In this case, as shown in FIGS. 7A and 7B, a depth ofthe potential well 14 formed in the charge holding period is smallerthan the depth of the potential well formed in the charge generationperiod. For example, the control voltage applied to the three controlelectrodes (1) to (3) or (4) to (6) to obtain the potential well 14having the large depth is 7V, and the control voltage applied to onlythe electrode (2) or (5) to obtain the potential well 14 having thesmall depth can be 3V. When the potential well 14 for mainly generatingthe electric charges (electrons “e”) has the larger depth than thepotential well 14 for holding the electric charges, the electric chargescan easily flow in the potential well having the large depth, so thatamounts of noises can be relatively reduced.

The electric charges provided from the charge ejecting portion 26 of thelight receiving element 2 is sent to the image generator 4. In the imagegenerator 4, a distance between a point of the object M and the imageprocessing device is determined by substituting the intensities (A0, A1,A2, A3) of the received light into the equation (1) with respect to eachof the photoelectric converters 20. As a result, 3D information aboutthe target space including the object is obtained. By using the 3Dinformation, a distance image having pixel values, each of whichprovides a distance value between a point on the object and the imageprocessing device can be generated.

On the other hand, brightness information of the object M can beobtained from the amounts of the electric charges provided from thecharge ejecting unit 26 of the light receiving element 2. That is, a sumof the amounts of the light received at each of the photoelectricconverters 20 or an average value of the amounts thereof corresponds toa gray value of the point on the object. As a result, a gray imagehaving pixel values, each of which provides the gray value of the pointon the object is obtained. In the present embodiment, to minimize theincident of outside light on the light receiving element 2, the lightsource 1 irradiates an infrared ray to the target space, and aninfrared-transparent filter (not shown) is disposed in front of thelight receiving element 2. Therefore, the gray image generated by theimage processing device of this embodiment is an infrared gray image.

Thus, both of the distance value between the point on the object and theimage processing device and the gray value of the point on the objectcan be obtained from the same pixel. Therefore, it is possible to obtainthe distance image and the gray image, which are substantially identicalin time. In addition, since each of the pixels of the distance image hasa one-to-one correspondence with each of the pixels of the gray image,no treatment of associating each of positions in the gray image withcorresponding distance information is needed. Moreover, greater spatialinformation about the object M can be obtained, as compared with thecase of using only the gray image.

The distance image and the gray image generated by the image generator 4are sent to the differentiator 50. In the differentiator 50, a distancedifferential image having pixel values, each of which provides adistance differential value, is generated from the distance image, and agray differential image having pixel values, each of which provides agray differential value is generated from the gray image. Each of thedistance differential value and the gray differential value can bedetermined by using pixel values of a center pixel in a predeterminedpixel region and neighbor pixels around the center pixel.

For example, as shown in FIG. 8, the distance differential value “Dd” ofthe center pixel p5 in a 3×3 arrangement of nine pixels (p1˜p9) of thedistance image is represented by the following equation (2).Dd=(ΔX ² +ΔY ²)^(1/2)  (2)

“ΔX” and “ΔY” are respectively obtained by performing the followingcalculations:ΔX=(B1+B4+B7)−(B3+B6+B9)ΔY=(B1+B2+B3)−(B7+B8+B9)Wherein, B1 to B9 are respectively pixel values of the pixels p1 to p9.Similarly, the gray differential value of the center pixel p5 of thegray image can be determined. In the distance differential image, as adistance difference in the distance image increases, the distancedifferential value becomes larger. Similarly, as a brightness (contrast)difference in the gray image increases, the gray differential valuebecomes larger.

Then, the distance differential image and the gray differential imageare sent to the outline extractor 52 to extract the outline of theobject M. In the present invention, it is preferred to extract theoutline of the object according to one of the following methods (1) to(5).

(1) A region(s) where the distance differential value maximizes in thedistance differential image, and a region(s) where the gray differentialvalue maximizes in the gray differential image are determined, so thatthose regions are extracted as the outline of the object.(2) A first region(s) where the distance differential value maximizes inthe distance differential image, and a second region(s) where the graydifferential value maximizes in the gray differential image aredetermined, and then a corresponding region(s) between the firstregion(s) and the second region(s) is extracted as the outline of theobject.(3) At least one of a region(s) where the distance differential value isnot smaller than a threshold value in the distance differential image,and a region(s) where the gray differential value is not smaller than athreshold value in the gray differential image is determined, and theregion(s) is extracted as the outline of the object.(4) A first region(s) where the distance differential value is notsmaller than a threshold value in the distance differential image, and asecond region(s) where the gray differential value is not smaller than athreshold value in the gray differential image are determined, and thena corresponding region(s) between the first region(s) and the secondregion(s) is extracted as the outline of the object.(5) A weighted sum of the distance differential value of each of thepixels of the distance differential image and the gray differentialvalue of a corresponding pixel of the gray differential image aredetermined, and then a region(s) where the weighted sum is not smallerthan a threshold is extracted as the outline of the object.

According to the above methods, a one-pixel width region including theoutline of the object can be extracted. In addition, according to themethod (1) or (3), it is possible to extract the outline of the objectwith a higher probability. For example, it is effective to extract inneroutlines or edges of the object. According to the method (2) or (4),even when there is a region having a large difference in brightness anda large change in distance in the target space, it is possible toaccurately extract the outline of the object, while preventing that anoise is extracted as the outline of the object by mistake.

In the methods (3) and (4), it is preferred that the threshold value forthe gray differential value is set to be different from the thresholdvalue for the distance differential value. In addition, there is anotheradvantage that the sensitivity of extracting the outline of the objectcan be controlled by changing a magnitude of the threshold value. Inparticular, when the region where both of the gray differential valueand the distance differential value are not smaller than the thresholdvalues is extracted, a remarkable effect of removing the noisecomponents is obtained. In the method (5), an order of precedencebetween the distance differential value and the gray differential valuecan be controlled by adequately setting weights used to determine theweighted sum. For example, when the weight for the distance differentialvalue is set to be relatively larger than the weight for the graydifferential value, a region having a large change in distance has ahigher priority as the outline of the object than the region having alarge change in brightness (concentration). In this case, for example,the outline of the human face can be easily extracted. It is preferredthat the image processing device further comprises a selector forperform a desired one from the above methods (1) to (5).

When a brightness difference (gray difference) between the object andthe background is small due to influences of outside light andreflection coefficient of the object, there is a case that the outlineof the object cannot be accurately extracted by use of only the graydifferential image In addition, when the distance difference between theobject and the background is small, it becomes difficult to extract theoutline of the object from only the distance differential image.However, according to the present invention, since the shortcomings ofthe distance differential image and the gray differential image arecomplemented to each other, it is possible to provide the imageprocessing device with an improved detection accuracy of the outline ofthe object. In addition, since the information obtained from each of thegray differential values and the corresponding distance differentialvalue is the information obtained at the same position on the objectfrom the same pixel, it is possible to prevent an oversight of theoutline to be extracted, and remove the noise components with a highdegree of reliability.

Second Embodiment

An image processing device of the second embodiment is substantially thesame as the device of the first embodiment except that an objectdetector 54 is provided in place of the outline extractor 52, as shownin FIG. 9. Therefore, the same components as the components of FIG. 1are indicated by the same numerals, and duplicate explanations areomitted.

The object detector 54 detects the object M according to the followingmethod with use of an output of the differentiator 50, i.e., a graydifferential image. The image generator 4 generates the gray image in atime-series manner. Therefore, a plurality of gray images are obtainedat different times. The object detector 54 generates a difference imagebetween a pair of gray differential images, which are generated from twoof the gray images, and then extracts a region where each of pixelvalues is not smaller than a threshold value in the difference image.The thus extracted region corresponds to a region of the object traveledin the target space. By the generation of the difference image, thebackground is substantially cancelled.

By the way, when a moving body other than the object M exists in thetarget space, it means that there is a noise component(s) in thedifference image. When the noise component(s) exists within a distancerange where the object M does not exist, it can be separated accordingto the following method. That is, a labeling treatment is performed tothe regions extracted from the difference image to obtain couplingregions. With respect to each of the coupling regions, an average of thepixel values of the distance image is determined, and then a regionwhere the average is within a predetermined range is extracted as theobject. Thus, by extracting a region corresponding to a desired distancerange, the noise components can be separated from the object.

To remove the background, the differentiator 50 may previously generatea reference gray differential image from the gray image obtained under acondition that no moving body exist in the target space. In this case, adifference image between the reference gray differential image and agray differential image obtained at a different time is generated. Byextracting the region where the pixel value is not smaller than thethreshold value from the difference image, the region of the object Mcan be easily separated from the background.

In addition, it is preferred that a stationary background other than theobject M traveling in the target space is cancelled by the followingmethod. For example, to extract the region corresponding to the object Mtraveling in the target space, electric charges are provided from thelight receiving element 2 such that the gray images are generated at aspeed of 30 frames/sec, as in the case of using a conventional TVcamera. The gray differential images are generated by the differentiator50 from the generated gray images, and then the difference image betweenarbitrarily selected two gray differential images is generated by theobject detector 54. When the object M travels at a relatively high speedin the target space, it is preferred to use the gray differential imagescorresponding to adjacent two frames to generate the difference image.

With respect to each of the pixels of the obtained difference image,which has a change in gray differential value between the graydifferential images, a pixel value other than zero is obtained.Therefore, by digitalizing the difference image by use of apredetermined threshold value, it is possible to extract the regionwhere the difference in gray differential value between the pair of theframes used to generate the difference images is not smaller than thethreshold value. Such a region corresponds to a region of the objecttraveled between two different times, at which the gray images weregenerated. Thus, by deleting the noise components, only the objecttraveling in the target space can be extracted. In this case, since theobject traveling in the target space is extracted from each of the twoframes, two regions corresponding to positions of the object at the twodifferent times appears in the digitalized image.

In the above case, the region of the object in one of the frames can notbe separated from the region of the object in the other frame.Therefore, when it is needed to extract only the region of the objecttraveling in the target space at a specific time (i.e., in a specificframe), it is preferred to perform the following treatment with use ofat least three frames obtained at different times.

For example, three gray differential images are generated from the grayimages obtained at three different times ((T−ΔT), (T), (T+ΔT)). Each ofthe gray differential images are digitalized by use of a predeterminedthreshold value, so that three digitized images (E(T−ΔT), E(T), E(T+ΔT))are generated, as shown in FIG. 10. In each of the digitalized image, aregion including the outline of the object M has a pixel value differentfrom the background. In this case, the pixel value of the regionincluding the outline of the object is “1”.

In the object detector 54, a difference between adjacent digitalizedimages E(T−ΔT) and E(T) in the time-series manner is determined.Similarly, a difference between another adjacent digitalized images E(T)and E(T+ΔT) in the time-series manner is determined. To determine thesedifferences, a logic operation EXCLUSIVE OR (XOR) is performed to a pairof each of the pixels of one of the adjacent digitalized images and acorresponding pixel of the other one. As a result, a digitalizeddifferential image are obtained from each of the differences. As shownin FIG. 10, two regions corresponding to positions of the object at thetwo different times appear in the respective digitalized differentialimage.

Next, a logic operation AND is performed to a pair of each of the pixelsof one of the digitalized differential images and a corresponding pixelof the other digitalized differential image. That is, since thebackground is substantially cancelled in these digitalized differentialimages, the region corresponding to the object in the target space atthe specific time (T) can be extracted by the logic operation AND. Thus,the result of the logic operation AND provides a silhouette at thespecific time (T) of the object traveling in the target space.

Subsequently, a labeling treatment is performed to the region obtainedby the logic operation AND to obtain coupling regions. With respect toeach of the coupling regions, an average of the pixel values (distancevalues) of the distance image is determined, and then a region where theaverage is within a predetermined range is extracted as the object. Inaddition, the regions existing in out of the predetermined range can beremoved as the noise components.

In the case of using more than three gray differential images, the abovetreatment may be carried out, as described below. For example, whenusing five gray differential images (1-5) generated from the gray imagesobtained at five different times, a logic operation AND between the graydifferential images (1, 2) is performed to obtain a resultant graydifferential image, and then a difference between the resultant graydifferential image and the gray differential image (3) is determined.Similarly, the logic operation AND between the gray differential images(4, 5) is also performed to obtain a resultant gray differential image,and then a difference between the resultant gray differential image andthe gray differential image (3) is determined. By performing the logicoperation AND between these differences, it is possible to obtain thesilhouette at the specific time of the object traveling in the targetspace.

Third Embodiment

An image processing device of the third embodiment is substantially thesame as the device of the first embodiment except for the followingcomponents. Therefore, the same components as the components shown inFIG. 1 are indicated by the same numerals, and duplicate explanationsare omitted.

The image processing device of this embodiment is characterized bycomprising an actual-size calculator 62 for determining an actual sizeof a desired portion of the object by use of the distance image and thegray image generated by the image generator 4, shape estimating unit 64for estimating a shape of the object M, and a volume estimating unit 66for estimating a volume of the object.

As shown in FIG. 11, the distance image generated by the image generator4 is sent to a measuring point determining unit 60. In themeasuring-point determining unit 60, a plurality of measuring points aredesignated in the gray image. The measuring points can be designated inthe gray image by the user. For example, it is preferred that the imageprocessing device further comprises a monitor 61 for displaying the grayimage, and a position designator 63 for allowing the user to designatethe desired measuring points in the gray image displayed on the monitor60 by touching a screen of the monitor, or by use of a pointing device(not shown) such as a mouse or a keyboard. In this embodiment, the grayimage displayed on the monitor 60 is an infrared gray image. As comparedwith the case of displaying the distance image on the monitor 60 anddesignating the measuring points in the distance image, positionalrelations between the object and the measuring points can be easilyrecognized by the user.

The measuring points may be automatically designated in the gray image.In this case, the image processing device comprises an object extractorconfigured to extract the object having a predetermined shape from thegray image generated by the image generator 4. Foe example, a positionof the object in the gray image can be determined by comparing the wholeshape of the object with a template. The measuring-point determiningunit 60 automatically designates a plurality of predetermined measuringpoints in the gray image in response to the shape of the object.

The measuring points designated on the monitor 61 and the distance imagegenerated by the image generator 4 are sent to the actual-sizecalculator 62. In the actual-size calculator 62, the distance value ofthe pixel corresponding to the each of the designated measuring pointsis determined from the distance image. In addition, positions of themeasuring points in the distance image are also determined. By using thedistance values and the positions of the measuring points, 3Dinformation about the measuring points on the object can be obtained.The actual-size calculator 62 determines the actual distance between twoof the desired measuring points by use of the obtained 3D information.The obtained actual size is displayed on the monitor 61. In addition, itis preferred that the obtained actual size is displayed by a straightline on the monitor. In the case of using the object extractor describedabove, the actual-size calculator 62 automatically calculates the actualdistance between two of the predetermined measuring points.

When at least three measuring points are designated, it is preferredthat plural pairs of adjacent measuring points are automatically set inthe order of the designation, and the actual-size calculator 62successively calculates the actual size with regard to each of the pairsof adjacent measuring points. For example, when three measuring points(m1, m2, m3) are designated, the actual-size calculator 62 successivelycalculates the actual sizes between the measuring points (m1, m2) andbetween the measuring points (m2, m3). In addition, when a plurality ofmeasuring points are designated along the outline of the object, it ispreferred that the measuring points corresponding to the maximum widthor the minimum width of the object are selected, and the actual-sizecalculator 62 calculates the actual size between the selected measuringpoints. In addition, when an outline of the object M is extracted by useof at least one of the gray image and the distance image, and themeasuring points are designated within a predetermined distance rangefrom the outline of the object, a treatment of replacing the measuringpoints on the outline of the object may be performed. For example, theoutline of the object can be extracted by use of the outline extractor52 explained in the first embodiment.

The shape estimating unit 64 is configured to estimate 3D informationabout a shape or an orientation of the object from at least one of thedistance image and the gray image generated by the image generator 4.That is, at least one of the distance image and the gray image is inputinto the shape estimating unit 64, and then edges (=outline) of theobject are extracted. As described in the first embodiment, theextraction of the edges is achieved by performing a differentialtreatment to the distance image or the gray image, and thendigitalizing. As the differential treatment, for example, an edge filtersuch as SOBEL filter can be used. The extracted edges are compared witha data base storing 3D information of given objects to determine as towhether they are components constructing the object.

In addition, when a plurality of candidates of the object exist in apredetermined distance range, it is preferred to determine whether theobject is integrally formed by those candidates. For example, when adistance difference between adjacent two candidates in the threedimensional space is not larger than a threshold value, these candidatesare determined as components constructing a single object. By using thenumber of pixels in a region surrounded by the candidates constructingthe single object and the distance value, the size of the object can beestimated.

The volume estimator 66 is configured to estimate a volume of the objectM in accordance with outputs of the shape estimating unit 64 and theactual-size calculator 62. In particular, it is preferred that themeasuring points are designated in the gray image, and a volume of aportion of the object defined by the designated measuring points isestimated by the volume estimating unit 66.

Fourth Embodiment

In this embodiment, a TV interphone using an image processing device ofthe present invention as an image pick-up camera is explained. The imageprocessing device is substantially the same as the device of the firstembodiment except for the following components. Therefore, the samecomponents as the components shown in FIG. 1 are indicated by the samenumerals, and duplicate explanations are omitted.

That is, as shown in FIG. 12, the image processing device of thisembodiment is characterized by comprising a reference-pixel detector 70configured to detect, as a reference pixel, a pixel having a minimumdistance value in a predetermined region in the distance image, and apixel extractor 72 configured to set a specific region including thereference pixel in the distance image, and extract a plurality of pixelseach having a distance value within a predetermined range from thespecific region.

For example, in FIG. 13, “E” designates a region defined by two dashedlines extending from the TV interphone 100. A distance image G1 of theregion E including the object M such as human can be generated by theimage generator 4, as shown in FIG. 14A In addition, “Qm” designate apoint providing a minimum distance between the TV interphone 100 and theobject M. A pixel Pm corresponding to the point Qm in the distance imageG1 is detected as the reference pixel.

Next, as shown in FIG. 14B, a specific region F is set in the distanceimage by use of the reference pixel Pm and the pixels each having thedistance value within the predetermined distance range extracted by thepixel extractor 72. For example, the pixels each having the distancevalue within the distance range defined between two dotted arcuate linesL1 and L2 in FIG. 13 are extracted by the pixel extractor 72. In FIG.14B, the pixels extracted by the pixel extractor 72 are shown by ahatching region. When a lower limit of the predetermined distance rangeis the distance value of the reference pixel Pm, and a value obtained byadding a required value (e.g., 10 cm) to the distance value of thereference value Pm is determined as an upper limit thereof, the objectconcerning the reference pixel (i.e., the object positioned at a minimumdistance with the TV interphone) can be extracted.

In addition, as shown in FIG. 12, the image processing device of thisembodiment is characterized by further comprising a gray-image memory 74for storing the gray image generated by the image generator 4, averagegray-value calculator 76 configured to read out the gray image from thegray-image memory 74, and calculate an average gray value of the pixelsof the gray image, each of which has the one-to-one correspondence withone of the pixels extracted by the pixel extractor 72, and an exposurecontroller 78 configured to control an exposure of the light receivingelement 2 in accordance with the obtained average gray value.

The exposure controller 78 controls an output of the light source 1 orthe sensitivity controllers 22 of the light receiving element 2 throughthe control unit 3, to provide an adequate exposure of the imageprocessing device. The reference-pixel detector 70, pixel extractor 72,average gray-value calculator 76 and the exposure controller 78 can beactualized by installing a required software in a microcomputer.According to the image processing device of this embodiment, theexposure can be automatically controlled to correct exposureirrespective of brightness of the target space or the condition of thebackground to clearly identify the object. Therefore, it is possible toprovide the TV interphone with an improved security performance.

As a modification of the above embodiment, a color image-pickup devicesuch as color CCD may be used as the image pickup camera. In this case,a color image is displayed on a TV monitor of the interphone, and theimage processing device described above is used to control the exposureof the color image pickup device.

Fifth Embodiment

In this embodiment, a TV interphone using an image processing device ofthe present invention as an image pick-up camera is explained. The imageprocessing device is substantially the same as the device of the firstembodiment except for the following components. Therefore, the samecomponents as the components shown in FIG. 1 are indicated by the samenumerals, and duplicate explanations are omitted.

As shown in FIG. 15, the image processing device is characterized bycomprising an alarm-mode setting unit 80 for setting an alarm modeagainst unwanted people, object extracting unit 82 configured to set analarm region in the distance image, and extract a group of pixels eachhaving the distance value within a predetermined range from the alarmregion as the object, characteristic-value extractor 84 configured toextract a characteristic value of the object extracted by the objectextracting unit 82, and human-body identifying unit 86 for determiningas to whether the object is a human body in accordance with thecharacteristic value extracted by the characteristic-value extractor 84,and an alarm reporting unit 88 for sending an alarm signal to a baseunit of the TV interphone when the object is determined as the humanbody by the human-body identifying unit 86. The object extracting unit82, characteristic-value extractor 84, human-body identifying unit 86and the alarm reporting unit 88 can be actualized by installing arequired software in a microcomputer. The alarm-mode setting unit 80 is,for example, actualized by use of a switch.

In FIG. 16, “Ra” designates an alarm region surrounded by two dashedlines extending from the TV interphone 100 and two dotted arcuate linesL3, L4. The pixels each having the distance value within the alarmregion Ra are extracted as the object by the object extracting unit 82.In the characteristic-value extractor 84, a pattern matching method isperformed by use of a suitable template to extract a portion having ahigh similarity as the characteristic value. Then, in the human-bodyidentifying unit 86; an area of the extracted characteristic value iscompared with a given threshold value to determine whether the object isthe human body.

According to the TV interphone of this embodiment, when a stranger comesin the alarm range Ra, the object M is extracted by the objectextracting unit 82, and the object is determined as the human body bythe human-body identifying unit 86. As a result, the alarm signal issent to the base unit of the TV interphone. On the other hand, when theobject other than the human body such as cat or dog comes in the alarmrange Ra, the human-body identifying unit 86 determines that the objectis not the human body. Therefore, the alarm signal is not sent to thebase unit of the TV interphone.

If necessary, the TV interphone described above may comprise a humansensor for sensing heat emitted from the human body such as pyroelectricinfrared sensor. In this case, since the control unit 3 of the imageprocessing device firstly receives an output of the human sensor, andthen the TV interphone starts to operate, it is possible to saveelectric power consumption of the TV interphone.

INDUSTRIAL APPLICABILITY

As described above, according to the present invention on theprecondition that a light intensity-modulated at a modulation frequencyis irradiated to the target space, and the light reflected from anobject in the target space is received by the light receiving element,the distance value and the gray value are generated from the electricaloutput corresponding to the intensity of the received light. Therefore,it is possible to obtain the distance image and the gray image, whichare substantially identical in time. In addition, since each of the grayvalues of the gray image and a corresponding distance value of thedistance image are obtained from the same pixel, there is an advantagethat no complex treatment of associating each of positions in the grayimage with the corresponding distance value is needed. The imageprocessing device of the present invention having the above advantageswill be preferably used in various applications such as a monitoringcamera for factory automation or a security camera for airports or otherfacilities as well as a TV interphone for home use.

1. An image processing device, comprising: a light source configured toirradiate a light intensity-modulated at a modulation frequency to atarget space; a light receiving element configured to receive the lightreflected from an object in the target space and generate an electricaloutput corresponding to an intensity of the received light; an imagegenerator configured to generate a distance image having pixel values,each of which provides a distance value between the object and the imageprocessing device, in accordance with a phase difference between thelight emitted from said light source and the light received by saidlight receiving element, and a gray image having pixel values, each ofwhich provides a gray value of the object, in accordance with theintensity of the received light; a differentiator configured to generatea distance differential image having pixel values, each of whichprovides a distance differential value, from said distance image, and agray differential image having pixel values, each of which provides agray differential value, from said gray image; and an outline extractorconfigured to extract an outline of the object by use of said distancedifferential image and said gray differential image.
 2. The imageprocessing device as set forth in claim 1, wherein said outlineextractor extracts, as the outline of the object, a first region wheresaid distance differential value maximizes in said distance differentialimage, and a second region where said gray differential value maximizesin said gray differential image.
 3. The image processing device as setforth in claim 1, wherein said outline extractor determines a firstregion where said distance differential value maximizes in said distancedifferential image, and a second region where said gray differentialvalue maximizes in said gray differential image, and then extracts acorresponding region between said first region and said second region asthe outline of the object.
 4. The image processing device as set forthin claim 1, wherein said outline extractor extracts, as the outline ofthe object, at least one first region where said distance differentialvalue is equal to or more than a threshold value in said distancedifferential image, and a second region where said gray differentialvalue is equal to or more than a threshold value in said graydifferential image.
 5. The image processing device as set forth in claim1, wherein said outline extractor determines a weighted sum of saiddistance differential value of each of said pixels of said distancedifferential image and said gray differential value of a correspondingpixel of said gray differential image, and then extracts a region wherethe weighted sum is equal to or more than a threshold value as theoutline of the object.
 6. An image processing device, comprising: alight source configured to irradiate a light intensity-modulated at amodulation frequency to a target space; a light receiving elementconfigured to receive the light reflected from an object in the targetspace and generate an electrical output corresponding to an intensity ofthe received light; and an image generator configured to generate adistance image having pixel values, each of which provides a distancevalue between the object and the image processing device, in accordancewith a phase difference between the light emitted from said light sourceand the light received by said light receiving element, and a gray imagehaving pixel values, each of which provides a gray value of the object,in accordance with the intensity of the received light, wherein saidimage generator generates said gray image in a time-series manner, andthe image processing device further comprises a differentiatorconfigured to generate a gray differential image having pixel values,each of which provides a gray differential value, from said gray image,and an object detector configured to detect the object by use of saidgray differential value and said distance value.
 7. The image processingdevice as set forth in claim 6, wherein said object detector generates adifference image between a pair of gray differential images, which aregenerated from two gray images obtained at different times, extracts aregion where each of pixel values is equal to or more than a thresholdvalue in said difference image, and then detects said region as theobject when a representative value of the pixel values of said distanceimage corresponding to said region is within a predetermined range. 8.The image processing device as set forth in claim 6, wherein said objectdetector generates a plurality of difference images, each of which is adifference between two of at least three gray differential imagesgenerated from at least three gray images obtained at different times,extracts a region where each of pixel values is equal to or more than athreshold value with respect to each of said difference images to obtainbinary images, performs a logical operation between each of pixel valuesof one of said binary images and a corresponding pixel value of anotherone of said binary images to extract a common region therebetween, anddetects said common region as the object when a representative value ofthe pixel values of said distance image corresponding to said commonregion is within a predetermined range.
 9. An image processing device,comprising: a light source configured to irradiate a lightintensity-modulated at a modulation frequency to a target space; a lightreceiving element configured to receive the light reflected from anobject in the target space and generate an electrical outputcorresponding to an intensity of the received light; an image generatorconfigured to generate a distance image having pixel values, each ofwhich provides a distance value between the object and the imageprocessing device, in accordance with a phase difference between thelight emitted from said light source and the light received by saidlight receiving element, and a gray image having pixel values, each ofwhich provides a gray value of the object, in accordance with theintensity of the received light; a reference-pixel detector configuredto detect, as a reference pixel, a pixel having a minimum distance valuein a predetermined region in said distance image; a pixel extractorconfigured to set a specific region including said reference pixel insaid distance image, and extract a group of pixels each having thedistance value within a predetermined range from said specific region;and an exposure controller configured to control a sensitivity of saidlight receiving element in accordance with the gray image having thepixels, each of which has a one-to-one correspondence with one of thepixels extracted by said pixel extractor.
 10. The image processingdevice as set forth in claim 9, wherein a lower limit of saidpredetermined range is the distance value of said reference pixel, andan upper limit of said predetermined range is determined by adding arequired value to the distance value of said reference pixel.